Methods are known from the related art with respect to 3D reconstruction using rolling shutter cameras (C. Geyer, M. Meingast, and S. Sastry. Geometric models of rolling shutter cameras. In Proc. 6th OmniVis WS, 2005 and G. Klein and D. Murray. Parallel tracking and mapping on a camera phone. In Proc. ISMAR '09, Orlando, October 2009), which compensate for the influence of the rolling shutter under the assumption of a known 3D geometry of the objects to be reconstructed (for example, fronto-parallel planes). More complex methods are also known from the publication “J. Hedborg, E. Ringaby, P.-E. Forssen, and M. Felsberg. Structure and motion estimation from rolling shutter video. In Proc. IEEE Int. Conf. on Computer Vision (ICCV) Workshops, 2011”.
A fundamental method for interpreting image sequences is the 3D reconstruction of the scene on the basis of at least two single images, which provide 2D projections of the scene from different camera positions. In this method, the camera movement is initially estimated from the image sequence data and the 3D coordinates of (static) spatial points are subsequently determined by triangulation (structure from motion, SfM). An important prerequisite for the exact 3D reconstruction of the observed scene is the sufficiently precise determination of the camera egomotion, i.e., of the camera pose (position and orientation) relative to the respective image recording times. At present, however, camera systems using rolling shutter (as compared to high-end camera systems using global shutter) are frequently employed in a variety of applications (driver assistance, safety engineering, robotics, mobile devices, etc.) for, among other things, costs reasons, so that the pixels of different scan rows are exposed and read out at different points in time.
This is generally not taken into consideration, however, when using SfM algorithms for the 3D scene reconstruction, from which systematic 3D reconstruction errors result. Methods, which compensate for the effect of the rolling shutter under the assumption of a known 3D geometry of the objects to be reconstructed (for example, fronto-parallel planes) form the related art with respect to 3D reconstruction using rolling shutter cameras. More complex approaches avoid the inclusion of a priori assumptions about the scene geometry by employing complex computation methods (bundle adjustment), which cannot be efficiently implemented in embedded camera systems (for example, a camera for driver assistance systems in the motor vehicle).
The approach presented herein enables an efficient implementation of a 3D scene reconstruction using rolling shutter cameras. This approach is suitable, in particular, for embedded systems having limited resources and enables a point-by-point 3D reconstruction with no systematic residual errors and requires no previous knowledge about the scene geometry.